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ABSTRACT 


Technological advances have made atmospheric mesoscale modeling at 
very fine resolutions readily available to a great number of organizations. 
Though initial operational results show some skill with respect to synoptic scale 
forecasts, many of the problems associated with mesoscale error growth and 
predictability have been ignored. Understanding mesoscale error is critical to 
accurately interpreting mesoscale model results and output from tactical decision 
aids (TDA’s). 

This study examines mesoscale error growth and predictability through 
controlled numerical model experiments. A Known “true” atmosphere is created 
through the use of the US Navy’s Coupled Oceanographic/Atmospheric 
Mesoscale Prediction System (COAMPS). Virtual observations are randomly 
sampled from this atmosphere to provide data for ingest into forecasts using the 
NCAR/Penn State MM5 mesoscale model. Forecast results for ten cases are 
compared against the “true” atmospheric solution and error statistics are 
calculated for wind speed and geopotential height fields. Results show how error 
growth and predictability are affected by different variables such as boundary 
conditions, weather regime, sample size and sample distribution. A scale 
separation of error is also performed in order to assess the impact of synoptic 
scale error on mesoscale error. 

Results indicate that error growth and predictability are dependent on all 
the above mentioned variables. Domain size and location define the 
wavelengths of error that are allowed to grow while boundary conditions limit 
which error grows within the domain. Contrary to previous studies, errors did 
grow on the mesoscale within a 24 hour forecast cycle. Scale separation results 


show that mesoscale error is dominated by synoptic sale error. 
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l. INTRODUCTION 


The vast number of technological breakthroughs in the last decade have 
allowed for enormous growth and opportunity In the field of mesoscale modeling. 
The increase in computer speed and power and the widespread availability of 
quality computing hardware has continued to fuel the vision that we can and 
should predict our environment at the finest resolutions imaginable. This growth 
has also engendered in some minds a belief that the finer the resolution, the 
better the forecast. While the recent growth in mesoscale modeling and 
increased resolutions has led to some useful insights into the key physical 
properties that govern mesoscale dynamics (Doyle 1997, Thompson et al. 1997), 
the general question of predictability and the influence that the synoptic scale 
has on the predictability limit of the mesoscale has been largely misunderstood 


or ignored. 


A. THE OPERATIONAL SIGNIFICANCE OF PREDICTABILITY 

The operational significance of the general predictability of mesoscale 
models and, in particular, specific mesoscale features cannot be overlooked. A 
good understanding of these concepts is crucial to effectively using today’s 
mesoscale model generated fields. The US Navy and other services are 


Increasingly relying on the accuracy of their mesoscale models not only to 


produce a detailed mesoscale forecast but also provide accurate information into 
sophisticated tactical decision aids (TDA’s). The output of these TDA’s can give 
detailed guidance to the warfighter who is trying to assess the environment 
he/she will be working in, but very rarely gives him/her a good picture of the 
reliability of that guidance. The model information is passed to the TDA and 
assumed to be accurate and relatively error free, yet there is no indication given 
that a particular mesoscale feature may not even be “predictable” at the temporal 
or spatial scale of the operation. While a good forecaster can and should 
accurately assess the character of the error in a particular model forecast 
through experience and comparison with other sources, there is currently no 
means by which a TDA can identify error. Further uncertainty in the guidance is 
brought about by the fact that very little, if any, actual mesoscale observations 
are assimilated into the mesoscale model. Thus, we must rely on the accuracy 
of the synoptic scale initial conditions to pass their dynamically significant 
features down to the mesoscale. At what scale then must we routinely observe 
the atmosphere in order to accurately forecast mesoscale weather events? And 
how accurate must we be on the synoptic scale in order to achieve a desired 
degree of accuracy on the mesoscale? Finally, how does the variability in 
sample size and location, mesoscale structure pattern, and boundary conditions 
affect error growth and ultimately predictability? It is with these questions in 
mind that we approached this work, with the ultimate desired result being a 


better understanding of error growth and predictability on the mesoscale. 


Bs WHAT !S PREDICTABILITY THEORY? 


The bulk of predictability work in the past two decades has been 
accomplished by Lorenz (1982, 1993) and Anthes (1984). Lorenz’ work focused 
on the concept of the predictability time limit, which represents the span of time 
between the best estimate of the state of the atmosphere based on initial 
observations and an estimate of its state at a future time. The predictability time 
limit represents the point where the forecast becomes essentially unusable due 
to inherent numerical model forecast errors. Beyond the predictability time limit, 
the likelihood of verifying the forecast is no better than random guesswork. The 
best estimate for the state of the atmosphere at the initial time depends first on 
the quality of the observational sample and then on the method of data 
assimilation. Based on Lorenz’ reasoning, the better this estimate, the longer 
the time before the predictability time limit is reached. Figure 1 shows a 
hypothetical change in numerical model forecast error (NMFE) over time and 
highlights the impact initial error has on predictability. At the initial time, the 
NMFE is small but grows rapidly due to the lack of dynamic consistency between 
the initial model state analyzed by observations and the dynamic allowances set 
forth in the model physics. As the model spins up, this initial state develops into 
a state that is more consistent with the models physical equations and thus a 
relative minimum occurs in the forecast error. Further development in the 
simulation shows that forecast error continues to grow due to the incompleteness 


of the model’s physical equations, its discretization, and truncation error. At 


some point, the NMFE reaches a critical level where the model essentially has 
no skill - the predictability time limit. 

The critical level of NMFE can be defined in various ways. In global 
atmospheric models, the critical NMFE is defined as the error variance (mean 
square of the difference of some variable) associated with two randomly chosen 
atmospheric states, or twice the climatological error variance (Anthes 1986). 
From the perspective of the warfighter, however, the critical level of NMFE is 
dependent on the nature of the operation which requires the numerical guidance, 
and may in fact be much lower. A third possible measure of the predictability 
time limit is the use of the standard deviation about the mean of a particular 
model variable. Though more a measure of the forecast skill of a particular 
solution, this measure also gives us insight into the predictability of the solution. 
lf the root mean square error of two numerical model solutions beginning with 
small differences at the initial time exceeds the standard deviation about the 
mean for a particular solution, then the other solution has in effect reached its 
predictability time limit with respect to the first solution. By calculating the mean 
of a certain variable such as wind speed in a control environment and then 
comparing the RMS error of a particular numerical solution to the standard 
deviation about the mean of the control, one can gain an accurate measure of 
the skill of that particular forecast solution. If the RMS value exceeds the 
standard deviation about the mean of the control, then in essence, the forecast 


has no skill, and simply taking the mean winds across the domain would provide 


a better forecast. We have utilized this third means of measuring predictability 
for all the error growth comparisons in this study. 

A similar predictability limit to the time limit can also be applied in space 
and is called the predictability spatial limit. It represents the spatial scale at 
which small scale variations become essentially randomly distributed and beyond 
which there is little useful information upon which to base a forecast. The 
distinction between the two predictability limits is that forecasts become less 
meaningful for increasing time scales, whereas they become less meaningful for 
decreasing spatial scales. Thus, as mesoscale modeling efforts seek to define 
increasingly smaller spatial scales over increasingly longer time scales, it 
becomes clear that understanding the predictability of certain features becomes 
even more important. 

The question of a theoretical predictability limit becomes more complex in 
that it often differs for different mesoscale phenomena. Although previous 
research suggested that predictability decreases rapidly with scale to limit 
mesoscale predictability to less than a day, common experience with mesoscale 
models shows that this is not always the case and that practical predictability 
time limits are often much longer. Baumhefner (1984) showed that for the 
synoptic scale, predictability decreases with decreasing scale at a typical error 
doubling time of about two days. Tennekes (1978) suggested that the 
mesoscale would reach the predictability time limit faster than the synoptic scale 


due to the transfer of energy from the small to larger scales by three dimensional 


turbulence. However, for the mesoscale phenomena that do not fit the average 
spectrum of atmospheric turbulence, the rate of this transfer of energy and thus 
forecast error, may differ considerably. Warner (1992) also suggested that some 
mesoscale phenomena, in particular those forced by fixed features such as a 
coastal mountain range, might be more predictable. Shukla (1984) found that 
since atmospheric variability is less in low lattitudes than in middle latitudes, the 
limit to deterministic predictability is shorter in the tropics than in the mid- 
latitudes, provided the error growth rates for a particular feature are the same. 
These results were all primarily obtained by performing predictability experiments 
on global models. This thesis will seek to examine and confirm a few of the 
results from Anthes, Baumhefner and Shukla’s experiments but also seeks to 
uncover some of the recent nuances of error growth on the mesoscale by 
utilizing current mesoscale model capabilities in a self-contained numerical 


modeling experiment. 


C. HYPOTHESES 


This thesis studies the dependence of mesoscale error growth and 
predictability on several different factors. Consequently, we suggest that the 
initial error growth as well as the predictability in a numerical forecast is strongly 


influenced by four factors: 


(1) The observational sample size, scale and distribution. 

(2) The method of data assimilation used to project information and error 
from observations onto the forecast model. 

(3) The choice of boundary conditions and frequency at which they are 
applied. 


(4) The characteristic structure of the mesoscale regime. 


Regarding factor (1), we seek to discover at what scale we must routinely 
observe the state of the atmosphere in order to accurately forecast mesoscale 
weather events. Regarding factor (2), we seek to understand how sensitive 
mesoscale forecasts are to both synoptic and mesoscale uncertainty in the initial 
conditions for a particular data assimilation system. Regarding factor (3), we 
seek to understand how boundary conditions impact the character of the error 
growth at all domain sizes. Finally, regarding factor (4), we seek to understand 
how the predictability time limit changes for different mesoscale regimes. Our 


specific hypotheses are that: 


1. Variation in the observational sampling pattern at the synoptic 
scale leads to variation in analyses that directly impact error 


growth and predictability in mesoscale forecasts. 


2. For a limited area model, the choice of boundary conditions and 
domain size directly impacts the character of the error growth by 
restricting the wavelength of allowable solutions. 

3. Error growth on the mesoscale correlates with error growth on 
the synoptic scale although errors grow more rapidly at smaller 
scales. 

4. Mesoscale predictability is weather regime dependent and error 
tends to be larger in scale and more variable in weather regimes 


with greater kinetic energy. 


D. EXPECTED RESULTS 


The expected results of this thesis are in essence a better understanding 
of error growth and predictability as it applies to the mesoscale. We will show 
that normal variations in observational sampling size and distribution, model 
horizontal grid spacing, as well as the system of data assimilation and boundary 
conditions all impact error growth and thus predictability. We will also estimate 
the shape of the NMFE growth curves as a function of these variables. By 
defining the critical NUFE amount as the standard deviation about the mean of 
our control atmosphere, we will evaluate the forecast skill and the predictability 
limit for different mesoscale regimes, hopefully showing that model forecasts 


initialized from better observational sampling regimes, and thus better analyses, 


will extend the predictability time limit for most mesoscale features. We also 
expect that at horizontal resolutions where there are little or no observations, the 
predictability time limit will be much shorter and mesoscale forecasts more 
suspect. 

Finally, we expect our results will produce more questions as we uncover 
the nature of the relationship of various factors to error growth on the mesoscale. 
Practically speaking, we will never be able to fully observe the atmosphere at the 
scales at which we desire to predict. The question that remains is then one of 
understanding and managing the error growth that we are certain will exist at 
those scales. By better understanding how synoptic scale variations impact the 
mesoscale domain, we may be able to develop techniques to limit these 
variations or at the very least, be able to identify when a mesoscale forecast is 


suspect and when it may be the right tool for the job. 
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ll. APPROACH 


One of the difficulties of performing atmospheric predictability studies is 
the inability to completely sample the entire atmosphere and thereby have a well- 
defined reference grid with which to perform the studies. Since operational 
observational sampling does not often capture true mesoscale structure, doubt is 
cast upon the accuracy of the error calculations for specific points within the 
domain. This study's approach therefore, utilized the benefits of a controlled 
environment using numerical model experiments. Although described in more 
detail in the following sections, the basic approach of our study follows. 

Instead of relying on operational sampling to describe a_ particular 
mesoscale regime, an idealized atmosphere was generated from a mesoscale 
mode! forecast. These model grids were then taken as the "true" atmospheric 
conditions for the course of the experiments providing the advantage of knowing 
the structure of the atmosphere at every point on the grid. Virtual observations 
were then developed by randomly sampling the ideal atmosphere in order to help 
assess the impact of observational sampling differences on both analyses and 
forecast. By combining the virtual observations with a synoptic scale first guess 
field, we performed several mesoscale model runs via a separate model in order 
to produce forecast fields for comparison. We could then determine the 
characteristic error in the model as well as the shape of the NMFE curves by 


comparing the analysis and forecast results to the “true” control initial 
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atmosphere. This procedure was performed for two separate weather patterns 
(winter and summer), and two distinct model predicted fields (geopotential 
height) and wind speed. Other sampling strategies included varying the number 
of observations in the analyses as well as running an experiment with no virtual 
observations, producing an analysis and forecast from initial guess points only. 
To understand the effects of boundary conditions on error growth, we also 
performed identical forecast experiments using two separate boundary 
conditions. In one experiment we utilized the Navy Operational Global 
Atmospheric Prediction System (NOGAPS) as the boundary conditions since this 
simulated the conditions actually used in Navy operations today, while in the 
other we utilized the "ideal" atmospheric conditions at the boundary to give us a 
sense of the "best" case forecast error growth. Finally, in order to understand 
how error might be passed down from one scale to another, we performed a 
scale separation of error for a single random case run. Throughout the testing 
phase, the main criteria for assessing the impact of the various sampling 
approaches on the forecast error growth was the change in forecast skill and the 
predictability time limit as described by the numerical model forecast error 
curves. Table 1 below summarizes the various experiments performed with the 


created data sets. 
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Summary of Experiments 


Design 




















Compared analyses of experiments using 0, 
100, 200, and 400 point samples to the “true” 
atmosphere and calculated RMS error. Noted 
trend in error with increasing point samples. 






1. Determine how varying sample size 
affects error growth. 


"Sampled" random observations from the "true" 
initial atmosphere, producing analysis and 
forecast for ten separate cases. Calculated 
RMS errors for wind speed at 500mb and noted 
the error growth differences between the 
various numerical solutions. 











2. Determine how various random distributions 
of observations affect error growth. 






. Determine how error growth is affected by 
varying mesoscale regimes 


Calculated error growth statistics for a definitive 
winter and summer pattern. 











Performed a scale separation of error by 
projecting larger scale fields onto 4km grid and 
then calculated the associated scale error. 





. Determine how mesoscale error is affected 
by larger scale error. 








Ran identical forecast simulations for both 
winter and Summer regimes using two different 
sets of boundary conditions. 






. Determine how choice of boundary 
conditions affects error growth. 


Table 1. Summary description of various experiments contained within the 
study. 


A. “IDEAL” ATMOSPHERE SPECIFICATION 


The idealized mesoscale atmosphere used as the basis for all the 
experiments and the random observations was created by running the US Navy's 
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) at 40 
vertical levels for a 36 hour "observing" period using four nested domains having 


horizontal grid spacings of 108, 36, 12, and 4 km centered over the San 
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Francisco and Monterey Bay region (Figure 2). A 12 hour data assimilation 
forecast was run for the 12 hours prior to the time period of initialization in order 
to spin up reasonable mesoscale horizontal and vertical structure along with 
dynamically consistent cloud and rain fields. The initial conditions for the data 
assimilation and the outermost domain's lateral boundary conditions originated 
from the 2.5 degree NOGAPS analyses for the corresponding time period. 

The experimental forecast fields used for comparison to the “true” 
atmosphere’s forecast were created using the NCAR/Penn State MM5 
mesoscale model. The model contained 30 vertical levels and was run with full 
physics. MM5 was chosen as the forecast model in order to allow for unique 
model solutions and to prevent reintroduction of error specific to the COAMPS 
model. Initial analyses on 12 vertical levels were created with NOGAPS first 
guess fields, and the randomly selected observations from the COAMPS “truth” 
mentioned above. The 12 level analyses were then interpolated to the 30 
vertical levels to initialize the MM5 forecasts. 

It should be noted that in order to provide a control forecast using MM5, 
the original COAMPS “true” atmosphere at reduced vertical resolution but with 
identical horizontal grid points was used for initialization. Since the experimental 
forecasts were run with MM5 starting from a 12 vertical level analysis (as in 
NOGAPS) , the MM5 control forecast also had to contain 12 vertical levels to 
provide fair estimates of error that were independent of the differing vertical 


resolutions. Although the control forecast contained less vertical structure than 
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the original COAMPS “true” atmosphere, the horizontal resolutions remained the 
same. The control forecast thus represented the best prediction that MM5 could 
achieve based on the COAMPS “true” atmosphere at consistent vertical 
resolution and was therefore a good basis for comparison to the experimental 
forecast runs. Figure 3 is a schematic diagram which summarizes the 


experiment’s design. 


B. RANDOM “OBSERVATIONS” 


In order to provide consistent amounts of observations for the initialization 
of the forecast runs, we utilized a random number generator to select random 
observations from the 108 km COAMPS “true” atmosphere. Ten cases, 
representing ten separate seed numbers into the random number generator, 
were run to provide some statistical significance to the study. 100 observations 
were selected randomly for each case run except for the sample size studies 
where we selected 200 and 400 observations respectively. The same seed 
numbers used in the winter case were also used for the summer case in order to 
provide consistency for comparison penvecn the two weather regimes. The 
forecast runs were thus initialized with NOGAPS global fields and the randomly 
selected observations added. Figure 4 shows the random distribution of 
observations for a specific seed number with 100, 200 and 400 observations 


respectively. 
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C. THREE DIMENSIONAL MULTIQUADRIC INTERPOLATION 


Three dimensional multiquadric(3DMQ) interpolation was used as the 
data assimilation method for these experiments due to its relative simplicity and 
the researcher's familiarity with the method. An added advantage of using the 
3DMQ approach was that it had the ability to resolve features at the smallest 
scales that were represented by observations in a particular region of the 
analysis domain (Nuss and Titley 1994). This ensured that we could accurately 
calculate the error associated with the finest mesoscale features. 

3DMQ uses radial basis functions which assume isotropic structure 
functions for the environment (Nuss and Titley 1994). This approach differs from 
the anisotropic error covariance structure functions of Optimal Interpolation (Ol) 
and three dimensional variational analysis (3DVAR) schemes. These data 
assimilation schemes rely on prior knowledge of the true state of the atmosphere 
in order to give an optimal analysis based on an estimated error covariance 
function. However, error in the estimates of the observation and background 
error covariance functions strongly impact the accuracy of the optimal fit. SDMQ 
interpolation is not affected by errors in covariance estimates since it does not 
rely on prior knowledge of the true state of the atmosphere. Theoretically then, 
there would be smaller potential error variations from one weather regime to 
another, allowing for consistent analysis between different weather regimes, 


which particularly suited the dual weather pattern approach of this experiment. 
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D. WEATHER REGIMES 


The idealized mesoscale atmosphere was generated for two distinct 
weather regimes, 1) a deep extratropical cyclone with an associated front 
interacting with the coastline occurring on February 6" 1998, and 2) a summer 
high pressure pattern which produced a typical sea breeze pattern in the 
Monterey Bay region, occurring on August 25", 1997. Figures 5[a-e] and 6[a-e] 
show the progression of each weather pattern for the 24 hour forecast period for 
the “true” atmosphere. Of primary importance in the differing weather regimes is 
the mesoscale structure associated with each regime and the amount of kinetic 
energy that each system contains. For instance, we expect that on the 4km 
nest, error growth associated with the winter pattern would be primarily 
influenced by the passage of the cold front and the model’s correct timing and 
Orientation of this event. For the summer pattern, however, we expect the error 
growth on the 4km nest to be associated more with the background flow and with 
the correct representation of the marine planetary boundary layer. Due to the 
greater amounts of energy associated with the winter pattern, we also expect 
that the magnitude of the errors associated with this system will be greater than 
those in the summer pattern. Another factor to consider is the start time of the 
models for the two weather regimes. The winter pattern begins at 00Z and thus 
the first 12 hours of the forecast are in the “cold” part of the forecast cycle. The 
summer regime however, started at 12Z and thus in this case, the first 12 hours 


of the forecast are in the “hot” part of the forecast cycle. 
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1. The Winter Regime 

Figure 5a shows the winter pattern beginning at 00Z on 6 February, 1998. 
A 987mb surface low pressure system is located in the Gulf of Alaska with an 
associated baroclinic frontal zone extending to the southeast. The low has 
begun occluding already at this point and a wave is forming along the frontal 
boundary at approximately 35N, 130W, which later develops into a new low 
pressure system. South of the wave, the frontal band extends to the southwest 
and is located approximately 450 nautical miles from the Monterey Bay, which 
encompasses the majority of the 4km domain. Surface winds in the vicinity of 
the Monterey Bay are southerly at 25kts. At 500mb, the 108km domain is 
dominated by a significant long wave trough whose axis lies between 135W and 
140W, notably behind the surface front. At 00Z, a vorticity maximum is entering 
the base of the trough and has an associated 105kt jet maximum. The front left 
quadrant of the vorticity maximum corresponds with the developing wave at 35N, 
130W. 

By six hours into the forecast, the Gulf of Alaska low pressure system has 
occluded and moved off to the northwest. The surface front extending from the 
new low pressure center at 35N, 127W is now about 300 nautical miles (nm) 
away while surface winds near Monterey have begun to turn to the southwest 
and have increased to 30kts. At the 500mb level, the vorticity maximum has 


begun to progress through the base of the trough and has taken on a more 
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northwesterly orientation. The trough axis is located at approximately 130W, still 
behind the surface front. The jet maximum has reduced in speed to 90 kts. 

Figure 5b shows the development of the pattern twelve hours into the 
forecast. The surface front has entered the 4km domain and has made landfall 
near the California/Oregon border, though it is still approximately 50 nm away 
from Monterey. Surface pressures in the vicinity of Monterey have decreased by 
8-10mb over the 12 hour period and surface winds have continued turning to the 
west while maintaining speed at 30 kts. The low pressure center is now located 
at 37N, 125W, just off the northern California coast. The upper level pattern 
shows that the vorticity maximum has continued to translate through the base of 
the trough and both the trough axis and the vorticity maximum are nearly in the 
same location as the surface front. The jet maximum has remained relatively 
constant. 

Sometime between 12 and 18 hours into the forecast, a significant event 
occurs which eventually is reflected in the error growth curves for the winter 
pattern. During this time frame, the surface front enters the western portion of 
the 4km domain and makes landfall in the vicinity of Monterey. By 18 hours 
however, the surface front has passed through Monterey and the other side of 
the 4km domain and surface winds have shifted completely to the west and 
northwest. The low pressure center has continued to move to the northeast and 


Is now located over the California/Oregon border with a central pressure of 


hg) 


996mb. At the 500mb level, the vorticity maximum has completely passed 
through the base of the trough indicating the trough has stopped deepening. 
Figure 5c shows the final progression of the winter pattern 24 hours into 
the forecast. The entire 4km domain and the majority of the other domains are 
under the influence of the subsidence behind the cold front and the atmosphere 
has become relatively stable again. At the 500mb level, the vorticity maximum 
has moved inland a considerable distance while the trough has filled. The jet 
maximum has also decreased to 70 kts and is now located well to the south of 


California. 


2. The Summer Regime 


Figures 6(a-c) show the 24 hour forecast for the summer pattern 
beginning at 12Z on 25 August, 1997. Unlike the winter regime, the summer 
pattern remains fairly dynamically consistent throughout the forecast cycle. The 
synoptic scale is dominated by an upper level trough to the north in the Gulf of 
Alaska that is also reflected at the surface as a filling low pressure system, and 
to the south by a weak high pressure ridge. There is a weak stationary front that 
separates the two systems that also does not move considerably during the 24 
hour forecast. 

The primary influence on the 4km domain is the presence of the weak 
high pressure ridge extending from just north of Pt. Conception to the southwest. 


The associated surface winds around the high set up a weak onshore flow and 
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favorable conditions for the development of thermally forced sea and land 
breezes in the Monterey Bay area. Although this case is a bit climatalogically 
atypical for the summer considering the location and strength of the trough at the 
coast, it was chosen for its stark contrast to the spatially and temporally dynamic 


winter pattern. 


E. NUMERICAL MODELING FORECAST ERROR (NMFE) 


Thus far, we have only alluded to the general concept of Numerical 
Modeling Forecast Error, its existence, and its impact on forecast accuracy. How 
this error is defined specifically, however, depends on many factors. This study 
will in part show how NMFE varies as a function of weather regime, sample size 
and distribution, boundary conditions, and horizontal model grid spacing. Since 
generating root-mean-square (RMS) error statistics was the simplest method for 
calculating NMFE with time, we utilized this approach as the primary method for 
determining the NMFE. We also calculated the bias errors for selected model 
parameters. The drawback associated with the simple RMS approach to NMFE 
is that it gave little insight into the source of the error; whether it was related to 
location, timing or intensity. To compensate for this shortcoming, we attempted 
to categorize the errors by other means in order to better quantify the NMFE for 
the above-mentioned factors. Another factor to consider was the fact that the 
area of study was located in a coastal environment where RMS errors were not 


uniformly spread across the domain but were rather correlated with the 
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topographic effects of the coastline. Thus, further error growth studies 
performed over a more uniform domain may show different results. 

For consistency in comparison throughout the experiments, we chose to 
calculate the NMFE using geopotential heights and wind speed and direction at 
900 mb. Although 500mb is less influenced by mesoscale forcing at the 
surface, this level was found to be representative of model error at all levels. We 
also calculated the average error throughout the column. Again, the character of 
the error between the various levels was fairly consistent and thus choosing a 
single level to represent error growth was representative of the actual error 
growth throughout the column. Finally, the specific definition of the critical NUFE 
with regard to the model horizontal grid spacing was the standard deviation 


about the mean of the chosen model parameter over the given domain. 


Fe SCALE SEPARATION OF ERROR 


To gain a better understanding of the effects of synoptic scale error on the 
mesoscale, we performed a scale separation of the error eniqull to each scale of 
the nested grid. The nonlinear nature of atmospheric processes allows energy 
exchange among all scales of motion (Anthes 1984) and thus uncertainty or error 
in any scale will eventually contaminate all scales. The design of the nested 
grids we have chosen for our study, however, are one-way nested and thus error 
can only propagate down scale. In other words, error on the 4km domain 


consists of error inherent to that scale as well as error propagated in from outer 


Ze 


scales while error on the 108 km domain consists only of error unique to that 
scale. We utilized the following equation to represent the total error for the 4km 


grid, 


Etotat = Eroskm + Esekm + Er2km + Eakm 


where the total error is the summation of the error unique to each individual 
scale. This same equation could be applied to the coarser grids as well by 
simply dropping the error term for the finer grid spacings. Thus, the total error at 


12km would be 


Erota = Esoskm + Esexm t+ Er2km 


and likewise for other nests. 

The first task was to represent each of the larger scale fields (108km, 
36km, 12km) for both the "true" atmosphere and the random case atmospheres 
on the 4km grid. Figure 7 shows the results of this procedure at the twelve hour 
forecast for a single winter regime case. The 4km domain representation of the 
larger scale wind fields are each shown on the 4km domain. As expected, the 
108km representation shows the long wave pattern while the other three 
representations show increasing amounts of mesoscale _ structure. By 
subtracting the true atmosphere’s 4km representation of each scale from the 
particular case atmosphere's 4km representation, we could systematically 


calculate the amount of error at the 4km grid scale that was due to other scales. 
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G. 


TERMINOLOGY 


To avoid confusion when referring to various model forecast runs, the 


following terminology and notation is used throughout the thesis: 


COAMPS “truth” refers to the original ideal atmosphere created for the 
purposes of random observational sampling, 

MM5 “control” forecast refers to the best representation of the “true” 
atmospheric progression through time and is used as the basis for 
comparison in all experimental forecasts. 

980206/00Z refers to the winter pattern while the summer pattern is 
970825/12Z. 

Analyses performed with COAMPS will be represented as simply 
“Analysis” while the initial forecast time for the MM5 control forecast (also 
an analysis) will be represented by FOO. 

All other forecast times will be represented in the same manner (i.e.F06, 
F12, F24, etc.). Thus, 980206/00ZF12 will be the winter pattern, OOZ run, 
12-hour forecast. 

For each weather pattern, we performed ten random case runs where we 
selected a seed number for the random number generator which then 
randomly selected “observations” on the outer domain. Each case is 


represented by its unique seed number. For example, c148954 is the 
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case whose seed number was 148954. The same seed numbers were 


used when sampling the winter and summer cases for consistency. 
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LE FINDINGS 


The broad scope of this project provided many avenues for exploration 
into the character of error on the mesoscale and the number of different ways to 
analyze the data was staggering. It was immediately apparent from the resulting 
error growth calculations that each of the previously mentioned factors impacted 
the error growth in different ways, individually as well as in combination. Figures 
8[a-d] and 9[a-d] show the complete 500mb wind speed error growth curves for 
all ten random cases at all domains for both weather patterns, and both 
boundary conditions. A cursory look at the differences between the various plots 
shows that differences in weather regime, boundary condition and scale all 
impact the character of the error growth as well as the skill and predictability of 
the forecasts. In some cases, these differences affect individual solutions and in 
other cases, the set of solutions as a whole. The findings for the specific error 
analyses performed will be presented here with a more in-depth discussion of 


the results presented in the following chapter. 


A. THE EFFECT OF SAMPLE DISTRIBUTION ON ERROR GROWTH 


Each random case of selected observations, when used in the 
initialization of a new model run, produced a distinctly different solution. Thus, 


within the confines of a particular weather pattern or boundary condition, the 


2/ 


error growth and predictability of each solution was strictly dependent on how 
well that particular set of observations characterized the structure of dynamically 


significant atmospheric features. 


1. Correlation between Analyses and Forecast Error 


The concept that a better analysis will produce a better forecast is one of 
the reasons for maximizing the amount of information by which a particular 
mesoscale model ts initialized. This thought is generally well accepted in the 
mesoscale modeling community and thus it also follows that if there is relatively 
low amounts of error at the analysis, there will be comparatively less error in 
successive forecasts (Anthes 1986). Our study found, however, that while this 
may be the case experientially, the error growth characteristics of the various 
cases show otherwise. Figures 10 - 13 are scatter plot diagrams of 500mb initial 
and 24 hour forecast wind speed errors for all cases in both weather regimes, 
using both sets of boundary conditions. Table 2 below lists the summarized 
correlation coefficients for the same scatter plots, but only for the 108km and 
36km domains. The values for the 12km domain are not listed since they did not 
accurately show the true correlation between the initial and forecast error. The 
temporal separation between FOO and F24 combined with the small domain size 
essentially meant that we were correlating two distinctly different parts of the 
weather pattern. As a result, the correlation coefficients showed a large negative 


correlation which did not fit with the positive correlation coefficients of the larger 
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scales. The values for the 4km domain are also not available since the forecasts 
all initialized from the same first guess fields with no observations present within 


the innermost domain. 


Winter Winter Summer Summer 
Por eae. B.C. << - B.C. wan oe B.C. | NOGAPS B.C. 
)108km | | 0.54382 54382 | 0.03931 03931 | 0.10863 10863 0.30328 


0.65788 0.386276 0.04164 0.01080 


Table 2. Correlation Coefficients between FOO and F24 forecast 500mb wind 
speed errors (all cases). 











Probably the first fact to notice is that there is not a great deal of 
correlation in general at any scale for either weather regime or boundary 
condition. This result might be expected due to the random nature of the various 
solutions and supports the argument for ensemble forecasting which seeks to 
take advantage of the random nature of various model solutions by perturbing 
initial conditions in its ensemble members. The result also suggests, however, 
that a low error at initial forecast time does not guarantee a low error at F24, 
even though many forecasters use this rule of thumb when analyzing initial 
forecast fields. 

secondly, these results also show that there is a greater amount of 
correlation between initial error and forecast error when the the MM5 “control” 
boundary conditions are used which suggests that the boundary conditions play 


a significant role in restricting the growth of various solutions away from their 
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initial dynamic states. Additionally, the results show that there is generally more 
correlation between initial error and forecast error during the the more 
synoptically forced winter pattern. This result also supports the impact the 
boundary conditions have on the forecast error of the mesoscale since the 
boundary conditions tend to act more upon those types of weather patterns that 
are synoptically forced. 

Though not as useful due to the fact initial fields for all ten cases were the 
same at the 4km scale, the 4km domain correlation measurements also present 
some interesting findings. For both weather regimes and boundary conditions, 
there appeared to be a grouping of solutions vice a natural spread as might be 
expected (Figure 13). Unlike the random spread seen at the larger scales, this 
result does not bode well for ensemble forecasting techniques at mesoscale 
domain sizes. Of course, we must take into account that the representative 
sample of solutions (10 per weather regime) is fairly small and larger data sets 


may prove otherwise. 


2. Asymptoting Error Values 


Following the reasoning that a better analysis produces a better forecast, 
another reasonable conjecture is that variance in the initial error for the ten case 
solutions would remain somewhat constant throughout the forecast and would 
match the variance of the solutions at F24. This however, was not the case. 


Figures 8[a,b] and 9[a,b] show that particularly at the 108 and 36km domains, 
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the initial widespread difference in the various solutions eventually disappear as 
the forecasts progress and that by F24, there is little difference in the error 
between the various solutions. In addition, the magnitudes of the errors in these 
cases seem to asymptote to a particular value, suggesting that there may be 
some constraint on the wavelengths at which the error is allowed to grow for 


these domain sizes. 


B. THE EFFECT OF SAMPLE SIZE ON ERROR GROWTH 


1. Increasing Sample Size 


As expected for both the winter and summer cases, when the number of 
sample observations was increased, the RMS errors in geopotential height 
decreased proportionally. Figure 14 shows the benefit of increasing the number 
of observations in the analysis of a particular case. For the winter pattern, the 
largest benefit is obtained when we the increase the number of observations to 
200 observations from 100 observations, although analysis error continues to 
decrease when 400 observations are used. Conversely, the benefit gained in 
the summer pattern is more consistent between 100, 200, and 400 observations. 
The rate of RMS error decrease was roughly the same at all levels although 
there were a few instances where a larger increase was found in the lower levels 


of the atmosphere. 
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Figures 15[a-d] and 16f[a-d] show the impact on NMFE when the 
respective analyses for 100 and 200 point samples are used to initialize the MM5 
model forecasts. The 400 point sample was not used to initialize MM5 due to 
analysis problems with some variables. The three curves on each plot show the 
900 mb error growth for case c148954 with 100 points, the growth curve when 
200 points are used, and the standard deviation about the mean wind speed, by 
which we are measuring forecast skill. The results showed that the increased 
observations impacted the error growth differently for each type of weather 


pattern, boundary condition, and domain size. 


a) Impact by weather regime 


Using the increased number of observations seemed to decrease 
the error for most scales more in the winter than in the summer, regardless of 
which boundary condition was used. For the summer regime, there was almost 
no benefit at all at the 108km domain (Figure 16a). This result suggests that the 
synoptically driven forcing of the winter pattern benefitted more from the added 
number of points since it was better able to define the synoptic scale features. 
Since the synoptic features were not as crucial during the summer, the benefit 
was not as great, particularly at 108km. There is some question about whether 
the improvement gained by increasing the number of observations in the winter 
pattern is significant. The answer to this question must take into consideration 


the specific requirements of the operations requiring the forecast information. 
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b) Impact by boundary condition 


For the winter case, the MM5 “control” boundary conditions 
provided more benefit than when the NOGAPS boundary conditions were used. 
For the summer pattern, there was little difference between the boundary 
conditions, neither of which seemed to have a significant impact. A particularly 
interesting result was that both sets of boundary conditions for both weather 
patterns seemed to negatively impact the error growth of the 200 sample point 
case when they were applied at F12. At the larger scales(108 and 36km), 
applying the boundary conditions seemed to increase error relative to the 100 
point sample, while at the smaller scales (12 and 4km), the 200 sample point 
case actually had greater error than the 100 point case! This was particularly 
evident for the summer pattern on the 4km domain (Figure 16d). This plot 
suggests that at least for this case, the initial benefit in the forecast gained by 
adding more observations is erased when the boundary conditions are applied. 
Similar experiments with the other cases and different weather patterns may 
prove otherwise. It should be noted that the influence of the boundary conditions 


should also decrease as one goes to the inner domains . 


c) Impact by domain size 


Comparing the impact of the increased sample size across the 
domain sizes showed that for the winter pattern, the decrease in error was about 


the same while for the summer pattern, there was a greater benfit gained at the 
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smaller scales. The character of the error at the larger domains remained 
basically the same between the 100 and 200 sample point cases. 
lt should be noted that at the 4km domain for both weather 
patterns, the standard deviation about the mean wind speed of the control is 
lower than the error associated with the forecasts (see Figures 15d and16d). 
Thus in essence, these forecasts have almost no skill. At this scale, the benefit 
of increasing the number of observations is related to the skill or predictability of 
the forecasts. For the summer regime (Figure 16d), the 100 sample point case 
has no initial skill and only gains skill around F12. The 200 sample point case 
however, has skill from about FO04 to about F16 when the error growth curve 
exceeds the standard deviation curve. The winter regime (Figures 15d) is even 
less promising as it essentially has no skill at any time in the forecast. 
These results point to the fact that increasing the number of sample points 
at initialization of a mesoscale model does not always guarantee less forecast 


error at the smallest nest, but does generally reduce error at larger scales. 


2. NOGAPS Firstguess Forecast 


Another experiment we performed related to sample size was to run the 
forecast model without any observations. The input into the model would thus 
only be driven by the NOGAPS first guess fields and the MM5 model physics. 
The results of this comparison are shown in Figure 17[a,b] for the winter case 


using the MM5 “control” forecast as the boundary conditions. As expected, the 
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forecast error associated with a solution containing no observations was much 
higher than those cases that were initialized with observations, particularly at the 
108 and 36km domains (Figure 17a). However, this was not true at the inner 
nests of 12 and 4km. Figure 1/b shows the 500mb RMS wind speed error for 
the winter regime at the 12km and 4km domains. Note that the firstguess 
solution shows no real skill at these domain sizes at any time ‘in the forecast 
whereas it does show relative skill at the 108 and 36km domains. Note also that 
for the first seven hours of the forecast, the NOGAPS firstguess solution has 
comparable errors to the solutions that were initialized with observations. By F08 
however, the firstguess solution dramatically increases in error and remains high 
throughout the rest of the forecast cycle. Thus, synoptic scale observations 
seem to have little benefit in the initial hours of a mesoscale forecast at the 


smallest nests of the mesoscale model. 


C. THE EFFECT OF LATERAL BOUNDARY CONDITIONS ON ERROR 
GROWTH 


One of the more surprising results of the various experiments we 
performed was the impact that varying boundary conditions had on the character 
of the error growth. In his early studies of predictability of a limited-area model, 
Anthes (1986) suggested that by supplying accurate lateral boundary conditions 
(LBCs) on the outer domain throughout a forecast, the error growth at the 


mesoscale would be limited. This result was realized when we used the MM5 
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control forecast as the boundary conditions for the MM5 forecasts. Figure 18 
shows the 500mb RMS wind speed error growth (or rather decay) curves for the 
winter and summer synoptic patterns at the 108km scale when the MM5 control 
forecast was used as the boundary condition. The downward trend in error is 
contrary to general predictability theory and the limiting error growth is true 
irrespective of the weather regime, being observed in both the winter and the 
summer patterns. Though not shown, a similar result was also found for the 
900mb RMS geopotential height errors. The cause of this trend can be 
explained by the requirement that the forecasts at set intervals (here every 12 
hours) meet the conditions of the boundaries provided by the “truth” thus 
eliminating the errors which may have propagated in from the boundaries at 
longer wavelengths. This result is supported by the observation that at the outer 
domain (108 km), both wind speed and geopotential height error growth curves 
for all ten case runs show a definitive convergence at the forecast times where 
the boundary conditions were interjected. This was followed by a divergence as 
the model resumed its characteristic deviation in solutions. 

Using the MM5 control as the boundary conditions for the forecasts was of 
course an indication of the lower bound of the error growth and so in order to 
provide a more realistic view of the error growth in operational settings, NOGAPS 
was used as the boundary conditions for the MM5 forecasts. Figure 19 shows 
the resulting 5010mb RMS wind speed error growth when the NOGAPS boundary 


conditions were used for the winter pattern. As shown, the character of the error 
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is significantly different than the forecasts which used the MM5 control boundary 
conditions. The upward trend in error growth is more representative of 
predictability theory and the realistic errors that can be introduced by inaccurate 
LBCs. Figure 20 shows the average RMS wind speed error growth (or decay) 
of the ten random solutions for the winter pattern. One might think of this plot as 
the ensemble average of the winter pattern forecast error. This plot shows how 
significant lateral boundary conditions are to the character of the error growth (or 
decay). Though the initial error is very similar, the error growth curves diverge at 
around FO6, and by F24 the average of the solutions between the two sets of 


boundary conditions show almost a 4 m/s spread in wind speed error. 


D. THE EFFECT OF MODEL SPIN UP ON ERROR GROWTH 


We observed that in the forecast runs for each case, the model required a 
certain amount of time to spin up the mesoscale structure not span et FOO. 
This was particularly evident at the 4km domain as seen in Figures 8d and Qd, 
where no mesoscale structure was present at FOO. By about FO6, there was a 
considerable spread in the solutions provided by each random case, essentially 
defining the maximum variance in the error for the forecast cycle. The 
requirement that the model “spin up” the mesoscale structure in the forecast by 
initializing directly from a synoptic scale model analysis or forecast is often 
termed a “cold start” at operational numerical forecasting centers (Hodur 1997). 


Monterossa (1999) documented the impact of cold starts on a mesoscale 
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forecast showing that in some cases, forecasts were actually better when a “cold 
start” was performed. The observable error growth using a previous 6 hour 
forecast, which is known as a “warm start,” was not performed during this study 
although comparing the results might prove interesting. 

Additionally, almost every error growth curve at all domains for both 
weather regimes and boundary conditions showed an initial downward trend in 
error, which is also attributable to the effects of the model spin up. Recalling the 
theoretical NMFE growth curve in Figure1, this initial downward trend reaches 
an error minimum before the forecast error begins to grow. This is the point at 
which the model’s initial state becomes consistent with the physical equations 
that govern the model’s behavior. Another interesting study for future research 
would involve manipulating the initial conditions in such a way as to achieve the 


lowest error minimum for the longest period of time. 


E: WEATHER REGIME DEPENDENCIES 


1. Kinetic Energy 


As expected, there also appeared to be a weather regime dependence in 
the error growth characterized by smaller error values for the summer case than 
for the winter case. The reason for this can be assumed to be the smaller 
amount of kinetic energy associated with the summer regime. The benefit gained 


from increasing the number of observations was also less distinct in the summer 
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regime than in the winter regime for the same reason. It should be noted, 


however, that the relative size of mesoscale error for both weather patterns when 





compared to the mean values of those patterns were similar. In other words, the 
summer pattern was not any more predictable than the winter pattern just 


because the magnitude of its errors were less. 


2. Unique Mesoscale structure 


We noted that for all the case runs, the type of mesoscale structure 
present seemed to impact the character of the error growth at certain times in 
the forecast runs. For example, in the winter case at the 4km domain, there was 
a sharp increase in 500mb RMS wind speed error at F18 followed by a sharp 
decrease in error. As mentioned earlier in the description of the winter weather 
pattern, the passage of the cyclone’ s cold front occurs around this same time. 
This trait is also reflected in the 500mb wind speed errors bias (Figure 21), which 
shifted significantly from positive pre-cold front to negative post-cold front (recall 
that bias estimates were performed by subtracting the random case from the 
“control” case). For the summer regime, the character of the error throughout 
the forecast cycle was fairly constant due to the less well defined mesoscale 
structure of this pattern (see Figures 9f{a-d]). Furthermore, there was no jump in 
error for the summer case primarily because the sea/land breeze feature that 
was present is thermally forced by the well known surface boundary and is 


Spatially fixed in the domain throughout the forecast. The winter frontal structure 
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however, is forced on all sides throughout the column and moves in space as the 
forecast develops. Thus, the winter pattern showed a more perceptible jump in 


error. 


E. DOMAIN SIZE DEPENDENCIES 

There was also a difference in the way that errors grew between scale 
sizes. In general, there was less error at the larger domain sizes while error 
magnitudes increased with decreasing scale sizes, reaching 10m/s for some 
solutions at the winter pattern’s 4km domain (Figure 8d). Again, this was more 
evident with the winter pattern than in the summer pattern. This result combined 
with the larger errors at the smaller scales suggests that mesoscale structure is 
more important at these scales. 

Note in Figures 8d and 9d that each of the random case solutions start 
with the same error value at FOO at the 4km domain. This is due to the fact that 
no observations were located within this domain size at initialization and thus 
each case was initialized with the same NOGAPS first aioe fields. After 
initialization, the error growth curves depart due to the impact of the observations 
at the larger scales. 

Comparing the random case solution’s error growth curves with the 
standard deviation about the mean of the control forecast shows that model skill 
decreases with scale. While all the model solutions show skill throughout the 


entire forecast at the 108 and 36km domains, some of the solutions begin to lose 
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skill at 12km in the winter pattern. At the 4km domain, there is little to no skill at 


all, regardiess of weather pattern or boundary condition. 


G. SYNOPTIC SCALE INFLUENCE ON THE MESOSCALE 

One of our hypotheses stated that variation in the observational sampling 
pattern at the synoptic scale leads to variation in analyses that directly impact 
error growth and predictability in mesoscale forecasts. Some of the more 
interesting results we found throughout the course of this work dealt with how 


synoptic scale error impacts mesoscale error. 


1. Synoptic and Mesoscale Correlation 


Contrary to our hypothesis and general consensus among forecasters 
(Gunderson 1999), there was not a great amount of correlation between the 
errors on the synoptic scale and those on the mesoscale. Figure 22 shows the 
correlation scatter plots of 500mb wind speed errors between the 108km and 
4km domains for both weather regimes and boundary conditions. Table 3 below 
summarizes the correlation coefficients between the synoptic scale (108km) and 
mesoscale (4km) 500mb wind speed errors. All random cases and all forecast 


times are included in these calculations. 
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10 cases x 24 fest times = 240 pts Winter Summer 


COAMPS B.C. 0.05694 0.407778 
NOGAPS B.C. 0.47736 -0.00923 


Table 3. Correlation Coefficients between synoptic scale (108km) 
and mesoscale (4km) 500mb wind speed errors 
(all cases, all forecast times) 





It is evident from these results that there is not much correlation between 
the synoptic scale and mesoscale errors regardless of weather regime or 
boundary condition. Even the winter pattern using NOGAPS as boundary 
conditions which has the greatest amount of correlation shows a rather bi-modal 
distribution on its scatter plot diagram. These results must be somewhat 
tempered since the RMS errors we calculated were calculated across the entire 
domain for each domain size. Thus, the low correlation values between the 


108km and 4km scales are expected. 


2. Scale Separation of Error 


The scale specific 500mb wind speed error for case c148954 on 
980206/00z2F12 is shown in Table 4. These values were obtained by first 
projecting the three larger scale fields from both the random case run and the 
MM5 “control” onto the 4km grid scale (Figure 7). We then created the 


difference fields by subtracting the scale specific fields of the case from the 
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control fields. RMS and bias errors were then calculated from the difference 


fields. 


RMS (m/s) BIAS (m/s) 


Table 4. Scale separation of 500mb wind speed error for 980206/00z2F 12 
(case c148954) 





As can be seen by the results listed in Table 4 above, the majority of the 
total error at the smaller scales is due to the characteristic error of the synoptic 
scale. The summation of the errors at each scale also shows, however, that our 
initial assumption about the independence of the errors at each scale was not 
completely correct. The calculated RMS error on the 4km domain which should 
be a sum of all the representative scale errors was 2.5 m/s while the summed 
value above, calculated by separating out the representative scales and then 
summing them is 4.28 m/s. This suggests that that the error at the smaller 
scales is not exactly additive and we have not accounted for all of the scale 
interactions actually present. This fact however, does not change the fact that 
the largest amount of error at the 4km scale is still dominated by 108km scale 


error present on the 4km domain. 
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IV. DISCUSSION 


Further examination led us to develop some possible reasoning behind 
the results we achieved from our experiments. Though limited to two case 
studies, the analysis of these results have shed considerable light on the 
character of numerical model error on the mesoscale and the limits of 
predictability on these scales. The discussion in this section will thus explain the 
results from the previous section in more detail and will tie key concepts 


together. 


A. CHARACTERISTIC WAVELENGTHS 


On the outer (108km) domain, the model error has a characteristic 
wavelength which is based on the domain size, location, and the distance 
between the observations. Numerical models can resolve features that have 
wavelengths of 2Ax or greater, where Ax is the observational spacing. Thus, a 
typical observational spacing of 200km for instance, will allow the model to 
resolve wavelengths on the order of 400km (Nuss 1998). Domain size and 
location is also important when considering a specific weather system as in this 
study. For example, in the winter regime, the 108km domain was situated in 
such a way as the be able to capture the entire long wave trough and its 


associated short wave pattern. Thus, error associated with timing and location of 
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that trough was fairly accurate when compared to a domain location that may 
have only captured a portion of the trough. 

Many of the findings of our study seemed to indicate that there were 
characteristic wavelengths of error that were allowed to grow while other 
wavelengths were unresolvable. The effect of the various domain sizes as well 
as the number of observations on the error growth both point to this conclusion. 
The value to which the error asymptoted at F24 for the outer two domains can be 
thought of as the error associated with the minimum resolvable wavelength at 
those domain sizes. We can expect this value to change with different 
combinations of domain size, location, and observational spacing. For example, 
increasing the number of observations in our study reduced the spacing between 
the observations (Figure 4), effectively reducing the size of the wavelengths that 


could be resolved. As a result, the magnitude of the error was reduced. 


Be LATERAL BOUNDARY CONDITIONS 


Related to the characteristic wavelength of the error due to domain size 
and location is the understanding that the different lateral boundary conditions 
imposed on the model in effect limited the wavelengths of the allowable solutions 
at the largest domain. The error growth at this domain then consists only of the 
error associated with the scales that could freely grow within the domain. For 


this reason, when the MM5 “control” forecast was used as the boundary 
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condition, the error for all cases actually decreased with forecast time, because 
the conditions at the boundary had to be met. Of course, this also occurred 
when NOGAPS was used as the boundary condition but since the NOGAPS 
conditions didn't maintain as strict a control over the true mesoscale structure of 
the system as the “control” conditions, certain wavelengths that were fixed under 
the “control” boundary conditions were allowed to grow under the NOGAPS 
boundary conditions. This resulted in the conventional error growth that we 
observed in Figures 8 and 9. 

The significant influence of the lateral boundary conditions on the outer 
domain and therefore the character of the error at all domains is further 
supported by the results of the scale separation experiment we performed. 
These results showed that the error at all domains is dominated by the error at 
the largest domain, which is primarily influenced by the boundary conditions. 

Recalling the result of the reversal of the error at F12 when 200 
observation points were used adds further insight into the impact of the boundary 
conditions. Increasing the number of observations from 100 to 200 points 
caused the dynamic structure of the field to be better defined, making it become, 
in essence, more sensitive to the impact of the boundary conditions. This, in 
turn, created a larger error when compared to the less sensitive 100 point case. 

The sensitivity of a particular regime to the boundary conditions is also 
confirmed when we compare the impact of the boundary conditions on the 


different weather regimes. The winter pattern, which was a synoptically forced 
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system, was particularly sensitive to the boundary conditions while the summer 
regime was not. The summer pattern on the other hand was dominated primarily 
by local forcing and had a less well-defined mesoscale structure. 

It is clear from the various ways in which we examined the effect of the 
boundary conditions on error growth, that lateral boundary conditions play an 
important role in the character of error at all scales. Especially for the case of a 
synoptically forced system such as the winter pattern in our study, it is more 
important to have accurate boundary conditions at the outer domain than to have 


lower initial error when trying to achieve the best mesoscale forecast. 


C. SYNOPTIC SCALE INFLUENCES 


One of the difficulties of understanding error at any scale is being able to 
get at the unique error associated with that scale while still maintaining a proper 
perspective of the influences other scales have on the total error. Though only 
performed for a single case, the findings from the scale separation experiment 
showed that total error in a mesoscale forecast is dominated by error at the 
largest scales. This may explain why common mesoscale forecasting 
experience has shown that even though errors are large at the mesoscale, 
mesoscale forecast fields look reasonable and are useful tools for the forecaster. 
Although the correlation measurements between the synoptic and the mesoscale 
error were not as definitive as we might have hoped, we must consider that we 


did not correlate the exact representation of the synoptic scale error on the 4km 
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domain with the error unique to the 4km scale. Rather, we correlated the error 
across the entire 108km domain with that across the significantly smaller 4km 
domain. When this is considered, the small correlation coefficients were a 
reasonable result. We should not expect that there would be a great deal of 
correlation between error across so large a domain with error on such a small 
domain. The understanding the forecaster must have however, is that synoptic 
scale error does impact the error at the mesoscale, but that the correlation of 
such error is only significant at the domain size of interest. Thus, even if a model 
seems to be handling the synoptic scale pattern well, if the synoptic scale error is 
large in the location of the mesoscale nest, there will be a great deal of error 
associated with the mesocale forecast. 

Our findings with the scale separation experiment also showed that error 
at each scale is not truly separable. The error at a specific scale is more likely to 
be the integral of error contributed by all the wavelengths represented. Since our 
assumption was based on a finite direrencing approach to the calculation of the 
total error based only on the scales included in the nested grid, our results did 
not completely match up with our expectations. Furthermore, even though the 
smaller nested domains are linked to the larger domains, they are still in essence 
separate forecasts. Thus, the larger scale fields we projected onto the 4km grid 


did not completely correspond at every grid point. 
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D. OBSERVATIONAL SAMPLE SIZE AND DISTRIBUTION 

The random distribution of observations in this study provided unique 
insight into the error variance associated with different model solutions. The fact 
that our findings suggest very little correlation between analysis error and 24 
hour forecast error was particularly enlightening. 

It was clear from the results of the scale separation experiment that the 
spread in the various cases at the smaller domains as seen in Figures 8d and 9d 
was due to the small differences at synoptic scales exciting particular mesoscale 
frequencies. Thus, the spread is in essence the measure of the sensitivity of the 
small scale error to synoptic scale differences. For our study, the winter case 
was more sensitive most likely because of its higher kinetic energy, more well- 
defined mesoscale structure, and the fact that it was a synoptically forced 
system. 

Increasing the number of observations from 100 to 200 points at 
initialization showed that more points does not always equal a better forecast, 
particularly at smaller scales. The added observations must define realistic and 


dynamically significant mesoscale structure in order to provide any benefit. 


= WEATHER REGIME DEPENDENCIES 


It was clear that there were some weather regime dependencies on 
mesoscale error growth and predictability. However, the weather regimes did not 


play as great a role in influencing the error growth as the boundary conditions 
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did. In general, the more mesoscale structure and kinetic energy a system has, 
the more likely it will be to have greater magnitude errors and the more likely it is 
to be influenced significantly by the accuracy of the boundary conditions. 
Weakly forced systems, such as the summer pattern in this study, are not greatly 
influenced by error at the boundaries. Initial error in the summer pattern was 
high and error remained saturated throughout the forecast cycle, particularly at 
the 12km and 4km scales. This suggests that we do have a great amount of skill 
at these scales, even at initial times in the forecast. 

Though we did not address specific mesoscale features directly 
influenced by topography, we may assume that the magnitude of the error 
associated with the summer pattern was less than that of the winter pattern in 
part because most of the mesoscale structure was forced by the surface. This 
fact requires that in order to ensure that mesoscale structure will be correctly 
forced, the model’s representation of the topography must also be accurate. 
Current mesoscale model topographies smooth out many significant features in 
addition to misrepresenting elevations (Monterrosa 1999), making the calculation 
of error at the smallest scales even more complex. Performing a study of the 
error associated only with the representation of the topography would be a 


worthwhile future endeavor. 
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V. CONCLUSIONS 


The results of this thesis have shown that although there have been some 
marked improvements in mesoscale forecasting, there are still some questions 
with regards to how accurate we can be at the smallest domain sizes. We 
showed that error growth is affected by many different variables, some of which 
we can control and others which we cannot, but all of which we must understand 
in order to provide accurate forecasts. Specifically, we showed how error growth 


on the mesoscale is dependent upon: 


e Lateral Boundary Conditions 
e Weather regime 

e Sample size and distribution 
e Synoptic scale error 


e Domain size 


For these specific model solutions, we also showed the relationship between: 
e Analysis error and forecast error. 
e Synoptic scale error and mesoscale error. 


e Unique error at different scales through scale separation. 
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A. MESOSCALE CERTAINTIES AND UNCERTANTIES 


Many of our initial expectations and hypotheses were confirmed by the 
results of the various experiments while other results were more surprising. 
Some results actually varied from previous work in this field. For instance, unlike 
Anthes’ (1986) results, which showed that 72-hour simulations in limited-area 
models are not sensitive to random uncertainties in initial wind, temperature, and 
moisture fields, our results have shown that even in a 24 hour simulation, error 
growth does occur at the 4km scale. 

The following statements reflect the summary of our findings and help to 
clarify those ideas which we hope will add to the understanding of mesoscale 


error growth and predictability. 


1. Lateral boundary conditions — LBC’s play a critical role in defining 
how error grows, particularly for weather patterns that are synoptically 
forced such as the winter regime in this study. The stronger the 
synoptic scale forcing, the more important the boundary conditions 
become in interjecting error or correcting error in the forecast. 

2. Weather patterns - For weather patterns that are weakly forced such 
as the summer regime in this study, error growth remains small 
throughout the first 24 hours. Initial error is large for these systems at 
ail scales and becomes quickly saturated throughout the forecast 


cycle. 
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Forecast skill - For these cases, there was little to no forecast skill at 
the 4km domain when comparing error to the standard deviation about 
the mean of the control field. 

Synoptic and mesoscale correlation - There is very little correlation 
between synoptic scale error and mesoscale error, hence, a good 
synoptic scale forecast does not guarantee a good mesoscale 
forecast. According to the scale separation of error, however, synoptic 
scale error dominates the error at the mesoscale. Thus, a poor 
synoptic scale forecast will also result in a poor mesoscale forecast, 


particularly if the error is located at the domain of interest. 


5. Analysis and Forecast error correlation - While correct sampling of 
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the atmosphere through sample distribution is critical to ensuring good 
analyses, there is very little correlation between initial error and 
forecast error. Thus, a good analysis does not guarantee a good 24 
hour forecast, particularly for weal) forced weather regimes. 

Sample size -_ Error is generally reduced when more observations 
are used at initialization, particularly at large domain sizes for weather 
regimes that are synoptically forced. Also, some observations are 
better than no observations when calculating error growth. Increasing 
the number of observations for a nested grid, can in some cases 


increase error at the smallest nest. 
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B. FURTHER RESEARCH 

The unique approach of this study and the relative ease of its 
implementation opened up a vast number of opportunities for further research. 
The advantage of using numerical experiments in a controlled “ideal” 
atmosphere cannot be overlooked. The consistency we were able to produce 
with our results and the certainty we gained by comparing the model experiments 
to an ideal atmosphere added to the validity of this research approach. The 
following suggestions represent possibilities for further research and study in the 


subject of mesoscale model error and predictability. 


1. Vary Weather Regimes and Geographic Locations 


Natural atmospheric variability as well as the growth rate of small errors 
varies with geographic locations and synoptic pattern (Anthes 1986). While this 
study provided a good representation of the typical weather regimes found in the 
winter and summer off the west coast of the U.S., performing this study for other 
weather patterns in other geographic locations would lend more credibility to our 
results. This is especially important since most of the error across the smaller 
domains in the nested model were linked to the topographic features common to 


the West Coast of the U.S. 
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2. Calculate Error Growth Using other Model Parameters 


While the error growth and predictability patterns we found were 
consistent for the two model parameters we studied (wind speed and 
geopotential heights), this may not be the case for all variables. Further results 
could easily be obtained for temperature and moisture parameters in the forecast 


model, which would add more depth to the results we have shown. 


3. Better Experimental Design 


Some constraints were placed on this study due to choices that were 
made early on and better experimental design would lead to more robust results. 
The most useful of these changes would be to represent the control and 
experimental forecasts at the full 40 levels within the MMS model. This change 
would result in more accurate error growth calculations. In addition, we could 
have calculated the true model error within the study and then subtracted it out 
from the total error. 

Another design change would include adding a second model initialization 
method. All of the results in this study were performed with a model initialized 
from a “cold” start. If a “warm” start initialization was also performed, we could 
compare the resulting error growth with the “cold” start results and draw some 


conclusions on which method produced better forecasts. 


of 


Thirdly, while the controlled environment of this study afforded great 
flexibility in design, the use of real time observations to calculate error would help 


to support some of the results we found. 


4. Error Growth Compared to Synoptic Forecast Over Time 

While we did perform a forecast experiment by using only the synoptic 
scale NOGAPS fields to initialize the mesoscale forecast, it would certainly be 
interesting to calculate error growth statistics for a completely synoptic scale 
forecast. This may answer the question: “Can a synoptic scale forecast actually 


be a better forecast than a mesoscale forecast?” 


5. Use of Spectral Analysis to Perform Scale Separation 


The scale separation experiment provided some interesting results which 
could easily be repeated for different weather regimes and forecast times. 
However, since our results showed that we were not able to capture the error 
specific to all wavelenghts, a spectral analysis of the error might be a good 


approach in performing scale separation in the future. 
6. Topography 


Topography played a major role in the character of the error throughout 


this study, yet the model topography within the model was much smoother than 
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reality. If the forecast error specifically due to topographic errors was able to be 


removed, a more accurate representation of error growth could be given. 
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APPENDIX A. FIGURES 


The following pages of figures are grouped together in this appendix in 


order to facilitate the reading of this work. 
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Figure 1. Theoretical NMFE growth curves 
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Figure 2. Domain sizes for the experiment’s nested 
grid. 
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Ideal Atmosphere Schematic 


Stepl. Original COAMPS truth at 40 levels 
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970825/00z 
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Random obs selected here (10 cases) 
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Step 2. MMS5 “control” at 12 levels (interpolated down from above) 
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Step 3. Experiments using MMS at 12 levels (10 cases) 
(NOGAPS 1* guess at 12 levels + random obs from COAMPS truth) 
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Step 4. Error Growth calculations 
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Error = “control” - experiment *geopotential height 
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Figure 3. Schematic diagram describing the design of the ideal atmosphere 
for the study. 
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Figure 4. Random distribution of 100,200, and 400 observations for case 
6148954. 
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Figure 5b. MM65 control F12 forecast for winter pattern. Fields are 500mb 
geopotential hts,vorticity, and winds (top) and surface pressure,temperature, 
and winds(bottom). 
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Figure 6b. MM5 control F12 forecast for summer pattern. Fields are 500mb 
geopotential hts,vorticity, and winds (top) and surface pressure,temperature, 
and winds(bottom). 
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Figure 7. 108, 36, 12, and 4km scale projections on the 4km domain gnid for 
the winter pattern (MM5 control). Fields are wind barbs and tsotachs at 500mb. 
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Figure 8b. 500mb wind speed error growth curves on 36 km domain for ten 
random case solutions when compared to the MM5 control forecast. Boundary 
conditions are NOGAPS (top) and MM5 conirol (bottom). 
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Figure 8c. 500mb wind speed error growth curves on 12 km domain for ten 
random case solutions when compared to the MM5 control forecast. Boundary 
conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 8d. 500mb wind speed error growth curves on 4 km domain for ten 
random case solutions when compared to the MM5 control forecast. Boundary 
conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 9b. 500mb wind speed error growth curves on 36 km domain for ten 
random case solutions when compared to the MM5 control forecast. Boundary 
conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure ¢ 9c. 500mb wind — error growth curves on 12 km domain for ten 
random case solutions when compared to the MM5 control forecast. Boundary 


conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 9d. 500mb wind speed error growth curves on 4 km domain for ten 
random case solutions when compared to the MM5 coniro!l forecast. Boundary 
conditions are NOGAPS (top) and MM5 contro! (bottom). 
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Figure 10. Correlation diagrams of 500mb wind speed errors between F0Q 
and F24 for the winter (top) and summer (bottom) patterns (108km domain) 
using MM5 control (left) and NOGAPS (right) boundary conditions. 
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Figure 11. Correlation diagrams of 500mb wind speed errors between FOO 
and F24 for the winter (top) and summer (bottom) patterns (36 km domain) 
using MM5 control (left) and NOGAPS (right) boundary conditions. 
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Figure 12. Correlation diagrams of 500mb wind speed errors between FQO 
and 24 for the winter (top) and summer (bottom) patterns (12km domain) 
using MM5 control (left) and NOGAPS (right) boundary conditions. 
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Figure 13. Correlation diagrams of 500mb wind speed errors between FOO 
and F24 for the winter (top) and summer (bottom) patterns (4 km domain) 
using MM5 control (left) and NOGAPS (right) boundary conditions. 
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Figure 15a. 500mb wind speed error growth curves on 108 km domain for 
case c148954 with 100 and 200 point initializations in the winter pattern. 
Boundary conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 15b. 500mb wind speed error growth curves on 36 km domain for case 
6148954 with 100 and 200 point initializations. Boundary conditions are 
NOGAPS (top) and MM5 control (bottom). 
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Figure 15c. 500mb wind speed error growth curves on 12 km domain for case 
6148954 with 100 and 200 point initializations. Boundary conditions are 
NOGAPS (top) and MM5 control (bottom). 
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Figure 15d. 500mb wind speed error growth curves on 4 km domain for case 
6148954 with 100 and 200 point initializations. Boundary conditions are 
NOGAPS (top) and MM5 control (bottom). 
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case c148954 with 100 and 200 point initializations for summer case. 
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Figure 16b. 500mb wind speed error growth curves on 36 km domain for case 
6148954 with 100 and 200 point initializations for summer case. Boundary 
conditions are NOGAPS (top) and MM5 contro! (bottom). 
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Figure 16c. 500mb wind speed error growth curves on 12 km domain for case 
6148954 with 100 and 200 point initializations for summer case. Boundary 
conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 16d. 500mb wind speed error growth curves on 4 km domain for case 
6148954 with 100 and 200 point initializations for summer case. Boundary 
conditions are NOGAPS (top) and MM5 control (bottom). 
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Figure 17a. 500mb wind speed error growth curves on 108 km (top) and 36 


km (bottom) domains for case c148954 with 100 point, 200 point, and 
firstguess only initializations. Plots are for the winter case with MM5 “control” 


boundary conditions. 
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Figure 17b. 500mb wind speed error growth curves on 12 km (top) and 4 km 
(bottom) domains for case c148954 with 100 point, 200 point, and firstguess 
only initializations. Plots are for the winter case with MM5 “control” boundary 
conditions. 
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Figure 18. 500mb RMS wind speed error growth curves on 108 km domain for 
ten random case solutions in the winter (top) and summer (bottom) patterns 
using MM5 “control” boundary conditions. 
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Figure 19. 500mb RMS wind speed error growth curves on 108 km domain for 
ten random case solutions in the winter (top) and summer (bottom) patterns 
using NOGAPS boundary conditions. 


of 


500 mb RMS wind speed error 


winter case (108km domain) 
NOGAPS B.C. 


~~—- NOGAPS 
BS. 


=~ VIM5 
Conirol 
BC 





0 


CPF OSPF OHH YM OL PH a F 


Forecast Time 


Figure 20. “Ensemble” average of 500mb RMS wind speed error growth 
curves on 108 km domain for all ten random case solutions in the winter 
pattern using both MM5 “control” and NOGAPS boundary conditions. 
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Figure 21. 500mb RMS (top) and Bias (bottom) wind speed error nn 
curves on 4 km domain for all ten random case solutions in the winter pattern 


using MM5 “control” boundary conditions. 
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Figure 22. Correlation between 108km and 4km 500mb wind speed errors for 
the winter (top) and summer (bottom) patterns using MMb5 “control” (left) and 
NOGAPS (right) boundary conditions 
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